Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction

Discovering relevant, but possibly hidden, variables is a key step in constructing useful and predictive theories about the natural world. This brief note explains the connections between three approaches to this problem: the recently introduced information-bottleneck method, the computational mechanics approach to inferring optimal models, and Salmon's statistical relevance basis.Comment: 3 pages, no figures, submitted to PRE as a "brief report". Revision: added an acknowledgements section originally omitted by a LaTeX bu

Statistical Complexity of Simple 1D Spin Systems

We present exact results for two complementary measures of spatial structure generated by 1D spin systems with finite-range interactions. The first, excess entropy, measures the apparent spatial memory stored in configurations. The second, statistical complexity, measures the amount of memory needed to optimally predict the chain of spin values. These statistics capture distinct properties and are different from existing thermodynamic quantities.Comment: 4 pages with 2 eps Figures. Uses RevTeX macros. Also available at http://www.santafe.edu/projects/CompMech/papers/CompMechCommun.htm

Structure and Randomness of Continuous-Time Discrete-Event Processes

Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical complexity of stochastic processes generated by finite unifilar hidden semi-Markov models---memoryful, state-dependent versions of renewal processes. Calculating these quantities requires introducing novel mathematical objects ({\epsilon}-machines of hidden semi-Markov processes) and new information-theoretic methods to stochastic processes.Comment: 10 pages, 2 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/ctdep.ht

Reductions of Hidden Information Sources

In all but special circumstances, measurements of time-dependent processes reflect internal structures and correlations only indirectly. Building predictive models of such hidden information sources requires discovering, in some way, the internal states and mechanisms. Unfortunately, there are often many possible models that are observationally equivalent. Here we show that the situation is not as arbitrary as one would think. We show that generators of hidden stochastic processes can be reduced to a minimal form and compare this reduced representation to that provided by computational mechanics--the epsilon-machine. On the way to developing deeper, measure-theoretic foundations for the latter, we introduce a new two-step reduction process. The first step (internal-event reduction) produces the smallest observationally equivalent sigma-algebra and the second (internal-state reduction) removes sigma-algebra components that are redundant for optimal prediction. For several classes of stochastic dynamical systems these reductions produce representations that are equivalent to epsilon-machines.Comment: 12 pages, 4 figures; 30 citations; Updates at http://www.santafe.edu/~cm

Diffraction Patterns of Layered Close-packed Structures from Hidden Markov Models

We recently derived analytical expressions for the pairwise (auto)correlation functions (CFs) between modular layers (MLs) in close-packed structures (CPSs) for the wide class of stacking processes describable as hidden Markov models (HMMs) [Riechers \etal, (2014), Acta Crystallogr.~A, XX 000-000]. We now use these results to calculate diffraction patterns (DPs) directly from HMMs, discovering that the relationship between the HMMs and DPs is both simple and fundamental in nature. We show that in the limit of large crystals, the DP is a function of parameters that specify the HMM. We give three elementary but important examples that demonstrate this result, deriving expressions for the DP of CPSs stacked (i) independently, (ii) as infinite-Markov-order randomly faulted 2H and 3C stacking structures over the entire range of growth and deformation faulting probabilities, and (iii) as a HMM that models Shockley-Frank stacking faults in 6H-SiC. While applied here to planar faulting in CPSs, extending the methods and results to planar disorder in other layered materials is straightforward. In this way, we effectively solve the broad problem of calculating a DP---either analytically or numerically---for any stacking structure---ordered or disordered---where the stacking process can be expressed as a HMM.Comment: 18 pages, 6 figures, 3 tables; http://csc.ucdavis.edu/~cmg/compmech/pubs/dplcps.ht

Occam's Quantum Strop: Synchronizing and Compressing Classical Cryptic Processes via a Quantum Channel

A stochastic process's statistical complexity stands out as a fundamental property: the minimum information required to synchronize one process generator to another. How much information is required, though, when synchronizing over a quantum channel? Recent work demonstrated that representing causal similarity as quantum state-indistinguishability provides a quantum advantage. We generalize this to synchronization and offer a sequence of constructions that exploit extended causal structures, finding substantial increase of the quantum advantage. We demonstrate that maximum compression is determined by the process's cryptic order---a classical, topological property closely allied to Markov order, itself a measure of historical dependence. We introduce an efficient algorithm that computes the quantum advantage and close noting that the advantage comes at a cost---one trades off prediction for generation complexity.Comment: 10 pages, 6 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/oqs.ht

Islands in the Gap: Intertwined Transport and Localization in Structurally Complex Materials

Localized waves in disordered one-dimensional materials have been studied for decades, including white-noise and correlated disorder, as well as quasi-periodic disorder. How these wave phenomena relate to those in crystalline (periodic ordered) materials---arguably the better understood setting---has been a mystery ever since Anderson discovered disorder-induced localization. Nonetheless, together these revolutionized materials science and technology and led to new physics far beyond the solid state. We introduce a broad family of structurally complex materials---chaotic crystals---that interpolate between these organizational extremes---systematically spanning periodic structures and random disorder. Within the family one can tune the degree of disorder to sweep through an intermediate structurally disordered region between two periodic lattices. This reveals new transport and localization phenomena reflected in a rich array of energy-dependent localization degree and density of states. In particular, strong localization is observed even with a very low degree of disorder. Moreover, markedly enhanced localization and delocalization coexist in a very narrow range of energies. Most notably, beyond the simply smoothed bands found in previous disorder studies, islands of transport emerge in band gaps and sharp band boundaries persist in the presence of substantial disorder. Finally, the family of materials comes with rather direct specifications of how to assemble the requisite material organizations.Comment: 7 pages, 3 figures, supplementary material; http://csc.ucdavis.edu/~cmg/compmech/pubs/talisdm.ht

Inferring Pattern and Disorder in Close-Packed Structures from X-ray Diffraction Studies, Part II: Structure and Intrinsic Computation in Zinc Sulphide

In the previous paper of this series [D. P. Varn, G. S. Canright, and J. P. Crutchfield, Physical Review B, submitted] we detailed a procedure--epsilon-machine spectral reconstruction--to discover and analyze patterns and disorder in close-packed structures as revealed in x-ray diffraction spectra. We argued that this computational mechanics approach is more general than the current alternative theory, the fault model, and that it provides a unique characterization of the disorder present. We demonstrated the efficacy of computational mechanics on four prototype spectra, finding that it was able to recover a statistical description of the underlying modular-layer stacking using epsilon-machine representations. Here we use this procedure to analyze structure and disorder in four previously published zinc sulphide diffraction spectra. We selected zinc sulphide not only for the theoretical interest this material has attracted in an effort to develop an understanding of polytypism, but also because it displays solid-state phase transitions and experimental data is available.Comment: 15 pages, 14 figures, 4 tables, 57 citations; http://www.santafe.edu/projects/CompMech/papers/ipdcpsii.htm

Inferring Pattern and Disorder in Close-Packed Structures from X-ray Diffraction Studies, Part I: epsilon-Machine Spectral Reconstruction Theory

In a recent publication [D. P. Varn, G. S. Canright, and J. P. Crutchfield, Phys. Rev. B {\bf 66}:17, 156 (2002)] we introduced a new technique for discovering and describing planar disorder in close-packed structures (CPSs) directly from their diffraction spectra. Here we provide the theoretical development behind those results, adapting computational mechanics to describe one-dimensional structure in materials. By way of contrast, we give a detailed analysis of the current alternative approach, the fault model (FM), and offer several criticisms. We then demonstrate that the computational mechanics description of the stacking sequence--in the form of an epsilon-machine--provides the minimal and unique description of the crystal, whether ordered, disordered, or some combination. We find that we can detect and describe any amount of disorder, as well as materials that are mixtures of various kinds of crystalline structure. Underlying this approach is a novel method for epsilon-machine reconstruction that uses correlation functions estimated from diffraction spectra, rather than sequences of microscopic configurations, as is typically used in other domains. The result is that the methods developed here can be adapted to a wide range of experimental systems in which spectroscopic data is available.Comment: 26 pages, 23 figures, 8 tables, 110 citations; http://www.santafe.edu/projects/CompMech/papers/ipdcpsi.htm